A Hebbian/anti-hebbian Network Which Optimizes Information Capacity by Orthonormalizing the Principal Subspace
نویسنده
چکیده
منابع مشابه
A Network Which Performs Orthonormalized Principal Subspace Extraction
We propose a linear neural network which nds the subspace spanned by the principal components of its inputs, and scales the outputs so that they are uncorrelated with equal variance. We suggest that, to a rst approximation, this orthonormalized principal subspace extraction will give improved transmission of information over other PCA networks in the presence of output noise. The network employ...
متن کاملA Subspace Network That Determines Its Own Output Dimension
Neural networks which extract the principal components or principal subspace of their input data optimize information extraction from noisy inputs. With noise on both input and output, a rst-order approximation is to extract the principal sub-space of the input, suppressing those input components with variance below a certain threshold value. We describe a 2-stage linear network which uses inte...
متن کاملA Hebbian/Anti-Hebbian Neural Network for Linear Subspace Learning: A Derivation from Multidimensional Scaling of Streaming Data
Neural network models of early sensory processing typically reduce the dimensionality of streaming input data. Such networks learn the principal subspace, in the sense of principal component analysis, by adjusting synaptic weights according to activity-dependent learning rules. When derived from a principled cost function, these rules are nonlocal and hence biologically implausible. At the same...
متن کاملA Comparison of Hebbian Learning Methods for Image Compression using the Mixture of Principal Components Network
A number of novel adaptive image compression methods have been developed using a new approach to data representation, a mixture of principal components (MPC). MPC, together with principal component analysis (PCA) and vector quantization (VQ), form a spectrum of representations. The MPC network partitions the space into a number of regions or subspaces. Within each subspace the data are represen...
متن کاملApproximating Optimal Information Transmission using Local Hebbian Algorithms in a Double Feedback Loop
Maximising mutual information (MI) under various constraints has been suggested as a goal for neural networks in a perceptual system. Networks using Hebbian algorithms have been found to be suitable for optimising MI with either input or output noise. In this paper we show that a double feedback loop network, using local Hebbian algorithms, can approximate the characteristics required for optim...
متن کامل